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ABSTRACT 

This paper describes the second part of a new three-part study of STEM and engineering design research - termed 
Design-Without-Make conducted by NC State University in 2009-10 in an NC high school. The hybrid quantitative, 
qualitative case study is being developed with the purpose of capturing what new technological learning occurs and 
how the new pedagogical learning benefits the technological learner. The first paper described a basic literature 
review, social learning theory, project-based learning study and the outline and aims of the study. This second paper in 
the sequence goes on to explain the main study and the major data results and findings for fhis new approach to 
teaching engineering design technologywithin STEM education. 

The main purpose of this study was: i] To assess whether students who participate in design-without-make activities 
achieve learning outcomes as successfully as or better than students of traditional design-with-make activities; ii] To 
determine student and teacher attitudes towards design-without-make activities within technology education. A one¬ 
way analysis of variance was conducted to evaluate the relationship between instruction and the change in pre- and 
post-test scores between groups. 

The study consisted of 27 non-random participants, with the control group having 10 participants and the treatment 
group having 17 participants. The mean score from both groups came to 14.37 (SD=5.43j, with a standard error of 1.05. 
The post-test scores ranged from 4 to 22 out of a possible score of 22. With F=2.04, p=.05, it was found there were no 
significant differences between the control and treatment post-test scores. 

Keywords: Project-Based Learning, Design-With-Make, Design-Without-Make, STEM Education. 


FRAMEWORK OF PAPER 

In a previous paper, the authors described the literature, 
and the learning framework relevance and scholarship to 
engineering design, (Thompson, Varnado, & Matthews 
(2009), A feasible direction for American Technology 
Education, i-manager's Journal of Educational Technology, 
VOL #5 Issue No. 4 April-June, 2009). 

In this paper the authors will review the main design-without- 
make study, and its pedagogical approach to teaching 
engineering design within STEM technology. It is their belief, 
that this student-centered, intuitive practice has a lot of 
potential to play an important role in moving forward STEM 
education. Barlex (2008) was also of this mindset, and in the 
International Journal of Technology and Design Education 
(2007), he heuristically suggests that while still a hands-on 
project-based approach, design-without-make saves time 


and money, and allows for greater student design 
creativity. And that it places more emphasis on teaching 
and learning the design process rather than focusing on 
mechanistic prototyping outcomes. 

INTRODUCTION 

Current technology education in North Carolina is built 
around the concept of 'hands-on' laboratory exercises that 
allow students to gain real-world experiences in 
developing, implementing, and evaluating technologies 
(NCDPI, 2006). These N.C. Technology Programs of Study 
consist of 14 core strands, eight pre-engineering strands 
and three visualization strands; the courses of study are 
located at: http://www.ncpublicschools.org/cte/ 
technology/curriculum/programs/. Specifically, these 
strands are used as specific courses of study in order to 
allow a student to explore basic and advanced 
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technological concepts and principles that can be related 
to a chosen career. 

Nevertheless, in the Pre-engineering Strand is the 
Introduction to Engineering Design core. And within this 
strand are the students objectives which are used to create 
a product based on the principles and elements of design. 
This Engineering Design strand is where Design-Without- 
Make was tested, as it fulfilled the requirements for this 
standard course of study. 

The state objectives which are listed below, were followed 
closely 

• Identify the principles and elements of design 

• Describe how the design process relates to 
technology and other disciplines 

• Create a product based on the principles and 
elements of design 

Prior Knowledge 

The high school students entering this design topic lesson 
were expected to have pre-requisite basic knowledge of (i) 
Technology (ii) its past, and (iii) what constituted a 
technological system before starting the lesson. Without this 
knowledge it would have been difficult for students to see 
how individual components in the system related to the 
part they were designing. Plus, prior to beginning this lesson 
students were expected to have read pages 145-147 and 
Chapter 9 in Technology: Today and Tomorrow by Brusic, et 
al (1999). They were also expected to reference their 
designs backto Chapters 7 & 9 from the class text. 

Teacher input and the process 

Teachers were provided with given lesson plans which 
included a design PowerPoint (Thompson, 2009). Within the 
lesson plans a teacher had to outline the basics of the 
design process to include: design definition, design brief, 
design principles, design process and design proposal. The 
teacher had to discuss the 6 elements of design: line, 
shape, form, texture/shade, and color. Teachers also had 
to discuss and explain the 7 principles of design: balance, 
proportion, contrast, variety, harmony, unity, rhythm. 
Included were the 8 factors of design: safety, reliability, 
economic consideration, quality control, environmental 
concerns, manufacturability, maintenance/repair, and 


ergonomics. In class it was expected that the discussion 
was centered on how the elements, principles, and factors 
of design work together throughout the design process and 
how each connected into Barlex's design pentagon, as 
seen in Figure 1. 

Design-Without-Make Activity 

At this juncture of teacher input, students were separated 
into random groups. Within the groups they were asked to 
research a new technology and to design a new 
implementation of that technology to solve a real world 
problem. Students had to follow the design process 
outlined in the PowerPoint (Thompson, 2009), which started 
with problem identification, and continued through 
brainstorming, rough sketching, and comprehensive 
layouts, and finished with presentation graphics. 
Sometimes teacher input was required to assist students in 
choosing a technology and problem. Finally, students then 
presented their designs to their peers for critique and 
evaluation. As shown in Table 1, a rubric was created to 
allow teachers to grade design results. 

Basic Methodology and Experiment Design 

The basic methodology for this blended study consists of 
both quantitative and qualitative components. In the 
quantitative component a non-equivalent quasi- 
experimental design is used. In the control group it used a 
traditional design-with-make class, while the treatment 
group consisted of a design-without-make class. Both 
groups were presented with equivalent pre- and post-tests, 
which were compared statistically using an ANOVA test. The 
qualitative data was collected in semi-structured teacher 
and student interviews. The whole experiment was based 


Conceptual 
What it does 



Figure 1. The design decision pentagon 
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Objectives 4 points 4 points 


12 points 20 points 

per 

objective 


Brainstorming 

Individual or 

Organization skills Evaluation of ideas 

and research 

group sharing 

and recording of upon completion 

of information 

of ideas 

ideas 

of gathering 
thoughts through 
more in depth 
research material 
with attention to 
references 

Thumbnails, 

Pencil/ ink 

Organization skills Finals layout of 

working 

layout/ 

and recording of thumbnails. 

drawings, or 

Mechanical/ 

ideas Layout and 

working drawings 

desktop 

CAD/ Desktop 

positioning of 

including line 

publishing 

Publishing 

drawings 

structure. 


format 


dimensions, details 
and notes 

Marketing 

Materials or 

Attention to 

Final production of 

Display 

software used 

details 

display including 


related to 


shape, form, 


display 


function, harmony. 


development 


and balance 

Technical 

Data 

Details of the 

Composite of all 

reporting or 

collection and design methods 

necessary 

portfolio 

processing of 

and procedures 

elements along 

development 

information 


with details within 
the portfolio 

Marketing 

Design 

Presentation 

Final presentation 

Presentation 

presentation 

methods 

of display in 


selections and 


relations to 


supply list 


following the 

Principles and 

Elements of Design 

Total Points: 

Comments: 


Table 1. Rubric to student group designs 


on social learning theory - and its relevance to engineering 
design. This is connected to Problem Based Learning 
practice of design-without-make which in turn is intrinsically 
linked to social learning theory, and design and creativity 
within the engineering design classroom as shown in the 
Table 2. 


SocialNetworking and Learning - Relevant to STEM 
Engineering Design 


Pertinent authors 
& scholarship 


Social learning theory 

In this digital age there are many forms of social learning 
especially when students are not limited to the classroom 
walls. 

Students often bond to each other through social 
interaction; learn from each other by close proximity; or 
learn by watching others working together. 

Modern digital savvy students also connect via social 
interaction outside the classroom. Students use email, 
twitter, facebook, blogs, virtual objects, and so on to 
communicate with each other. 

Social learning theory focuses on the learning that occurs 
within a social context. Thus, students learn and reason 
verbally and the teacher / mentor / parent acts as a 
collaborator to provide positive and appropriate social 
interaction in order to learn from each another. 

The theory considers that through social interaction 
students learn from one another. It includes such 
concepts as observational learning, imitation, 
self-efficacy, self-regulation, motivation and 


Bandura (1977); 

Vygotsky (1978); 

Davydov & 

Kerr (1995); 

John-Steiner 

( 2000 ); 

Murphy & 
Hennessy (2001); 

Trebell (2007); 

Gredler (2005) 

Marton & 

Saljo (1976 


SocialNetworking and Learning - Relevant to STEM Pertinent authors 

Engineering Design & scholarship 

modeling (live and/or symbolic modeling). 

Bandura directed his initial research to the role of social 
modeling in human motivation , thought, and action. 

Albert Bandura is considered one of the leading 
proponents of this theory, but Bandura never looked at 
digital connections as communication. 

Design and Creativity 

Sternburg cites it as a method of educating students to 
effectively use knowledge and skills in what is called 
educating for wisdom. (Wisdom being made up of a 
combination of intelligence, creativity, and wisdom, 
which in turn is influenced by one's personal value system 
on interpersonal, intrapersonal, and extrapersonal levels. 

Spendlove uses the Intrinsic Motivation Theory Principle of 
Creativity. He defines that intrinsic, emotionally engaging 
activities are highly conducive to creative acts. He also 
lists five 'sure-fire' killers of creativity, which are: expected 
reward, expected evaluation, surveillance, time limits and 
completion. 

Problem - and Project-based learning 

A method of teaching problem-solving skills in which 
students work together as they progress through a series 
of steps to design, implement, and evaluate solutions to 
real world problems. 

In both Problem- and Project-based learning (PBL), the 
teacher provides complex tasks based on challenging 
questions or problems that involve the students' problem 
solving, decision making, investigative skills, and reflection 
that include teacher facilitation, but not direction. 

The teacher is more a facilitator and more focused on 
questions that drive students to encounter the central 
concepts and principles of a subject hands-on. 


Design-with-make 

Design-with-make uses six basic steps in every Technology 
Education design-with-make activity. They are 1. Identify 
and clarify problems; 2. Conduct research which might 
involve investigations; 3. Generate one or more design 
proposals; develop these so that they can be scrutinized 
for predicted performance and social/environmental 
impact; 4. Construct a prototype of the most promising 
design; 5. Experiment with sub-component designs as 
necessary; 6. Test/evaluate the constructed solution. 

Design without-make 

The methodology allows students to come into contact 
with creative design experiences without emphasis on 
building or project construction. 

A design-without-make activity is designed around six 
key concepts. These are 1. The Students design, but do 
not make; 2. They design products and services for the 
future; 3. They use new and emerging technologies in 
their design proposals; 4. They write their own design 
briefs; 5. They work in teams/ groups"; 6. They present 
their proposals to their peers, teachers, and mentors 
and to adult audiences at innovative conferences. 

Teachers are encouraged to challenge students with 
design-without-make activities which forces students to 
design products based on conceptual (what it does), 
technical (how it works), aesthetic (what it looks like), 
constructional (how it fits together), and marketing 
(who it's for) criteria without actually having to 
manufacture a final product for grading. 

Table 2. Social Learning Theory - Relevance to 
Engineering Design Literature 

Social Learning Theories - Review of Literature 

Success in education can be directly linked to research 
done in social learning by Vygotsky (1978), Davydov (1995), 
John-Steiner (2000), Murphy & Hennessy (2001). Trebell 


Barlex (2007); 

Barlex & 

Trebell (2008); 

Banks & 

Jackson (2007); 
Atkinson (2000); 
Peterson (2001) 


Spendlove 

(2007); 

Kipperman & 
Sanders (2007); 

Trebell (2007); 
Badran (2007); 
Schwartz (2007) 


Mills (2003); 

Banks and 
Jackson (2007); 

Matthews (2004); 

Albanese & 
Mitchell (2003); 

Lambros (2002) 


Sternberg, 
Reznitskay & 
Jarvin (2007); 

Standards for 
Technological 
Literacy; ITEA; 

Badran (2007); 

Spendlove (2007) 
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(2007) states that designing is a "social activity drawing on 
interaction between pupil/pupil and pupil/teacher" (p. 2). 
Teaching tools such as scaffolding and group work are very 
important in the social learning classroom. Vygotsky (1978), 
states "[t]he tasks that the child can accomplish in 
collaboration with the teacher today, she can accomplish 
alone tomorrow" (as cited in Gredler, 2005, p. 324). He 
encourages teachers to challenge students by designing 
lessons that kept students in their Zone of Proximal 
Development, which is a level of performance just above 
what the student can achieve on their own, but not more 
than they can achieve with the teacher's help. He 
concluded that students learn and reason verbally and 
that the teacher was only there to provide the social 
interaction which the students needed since students learn 
through social interactions with knowledgeable members 
of culture (Gredler, 2005). 

Design-With-Make 

Traditionally technology educators have used design-with- 
make projects to enhance, encourage, and allow for 
creativity among its students. After all, when students are 
provided with LEGO™ robotics programming modules, 
they can easily create and develop interactive storylines 
and props to accompany any discipline; even literature 
(Berg et. al, 2008). However, research has shown that, 
"[pjoor practice with education is often focused for reasons 
of expediency on the product stages of the creative 
process and in doing so bypassing the essential creative 
(person) and learning (process) elements and resulting in 
embellished, rather than creative, novel and inspiring, 
outcomes with limited contextualized learning, emotional 
engagement or opportunities to engage in risk taking and 
uncertainty" (Spendlove, 2007, p. 53). 

Kipperman and Sanders (2007) outline six basic steps in 
every Technology Education design-with-make activity. 
They are "i). Identify and clarify problems; ii). Conduct 
research which might involve investigations; iii). Generate 
one or more design proposals; iv). Develop these so that 
they can be scrutinized for predicted performance and 
social/environmental impact; v). Construct a prototype of 
the most promising design, experimenting with 
subcomponent designs as necessary; and vi). 


Test/evaluate the constructed solution" (p.227). They also 
recommend that "during this process the students should 
document all design, construction and testing procedures" 
(Kipperman, 2007, p. 227). 

Design-Without-Make 

A design-without-make activity is designed around six key 
concepts. These are "pupils design, but not make"; "pupils 
design products and services for the future"; "pupils use 
new and emerging technologies in their design proposals"; 
"pupils write their own design briefs"; "pupils work in groups"; 
and "pupils present their proposals to their peers, teachers 
and mentors and to adult audiences at innovation 
conferences" (Barlex & Trebell, 2008, p. 124). In their article, 
Design-without-make: Challenging the conventional 
approach to teaching and learning in a design and 
technology classroom, Barlex and Trebell (2008), define 
creative activities as "having four characteristics: (i), 
imaginative thought or behavior, (ii) purpose, (iii) originality 
(new to the creator) and (iv) an outcome of value" (p. 121). 
They also acknowledge that to develop creativity, "children 
must be actively involved in the learning process...[and] 
group work and collaboration are now seen as key 
elements" (Barlex & Trebell, 2008, p.l 21). Barlex and Trebell 
encourage teachers to challenge students with design- 
without-make activities which force students to design 
products based on conceptual (what it does), technical 
(how it works), aesthetic (what it looks like), constructional 
(how it fits together), and marketing (who it's for) criteria 
without actually having to manufacture a final product for 
grading (Barlex, 2007, Barlex & Trebell, 2008). 

Design-without-make activities work well in creative learning 
environments as defined by Isaksen (1994). He concludes 
that the more challenge, freedom, support, trust, prestige- 
free discussions, humor, and risk-taking the individual 
perceived in the immediate social work environment the 
more opportunity students have to be creative. This 
description is closely linked to the beliefs Barlex (2007) 
identifies as necessary for teachers who wish to host design- 
without-make activities in their classroom. He says teachers 
who believe "students intellectual abilities are socially and 
culturally developed"; "tasks need to be culturally 
authentic"; "prior knowledge and cultural perspectives 
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shape new learning"; "learners construct rather than 
receive meaning"; "pupils share responsibility for learning 
with teachers"; and "pupils are motivated by dilemmas to 
which they are emotionally committed" (Barlex, 2007, 
p. 156), will be most successful at integrating design- 
without-make activities, 

Banks and Jackson (2007) point out how, despite many 
students being motivated to take technology courses 
because of the hands-on process of physically making a 
product, these physical artifacts often lack any creativity or 
innovation on the point of the student, due to teacher 
designed plans. While these projects are easy to 
implement and fun for students to complete, often they are 
evaluated based on the completion of the product and an 
accompanying portfolio activity. In his research, Atkinson 
(2000) describes how when it comes to portfolio evaluation 
teachers tend to reward 'thin' evidence before rewarding 
students for exhibiting higher-order thinking skills. Barlex 
(2007) in a series of interviews with students in design and 
make classrooms learned that students tend to develop 
design portfolios after the product has been completed, 
which undermines the entire portfolio activity. When an 
entire class of students' products are identical in 
appearance and they are not performing the proper 
design and problem solving processes during the creation 
phases of said products, are they putting innovation in 
action? More simply stated, are these students studying 
technology effectively? 

This is the case for implementing design-without-make 
activities into Technology Education classrooms. Barlex's 
research has "revealed that pupils can be successfully 
engaged in designing without attendant making and that 
the current use of the portfolio for assessment purposes is 
for many pupils a highly demotivating experience" (2007, 
p. 160). He attributes this 'demotivation' to the fact of 
students not recognizing the value of the portfolio due to 
the way in which it is ineffectively implemented with the 
project, while he also points out that "the advantages of 
collaboration between pupils can be lost when there is an 
overemphasis on making" (Barlex, 2007, p. 160). 
Advantages of Design-Without-Make 
The implementation of design-without-make activities in 


place of some design-with-make activities within the 
Technology Education classroom has many advantages. 
First, design-with-make is often approached as if the act of 
designing is a linear process, rather than an inter¬ 
connected, reflective, non-linear series of steps (Barlex, 
2008), Figure 1 shows Barlex's design decision pentagon, 
which demonstrated the interconnectedness of the 
elements within the non-linear approach to designing 
taught in design-without-make activities. Second, group 
work and active involvement in the learning process, and 
risk-taking are all encouraged in design-without-make 
activities (Barlex, 2008, Trebell, 2007). These happen to also 
be important aspects of a creative learning environment, 
which is necessary for students to be innovative designers. 
A third major advantage of design-without-make in the 
classroom is the lack of large amounts of physical 
resources required in traditional design-with-make 
activities, such as: tools, equipment, and consumable 
materials (Barlex, 2007). 

Research Questions 

The research questions this study sought to address include: 

• Did students who participated in design-without-make 
activities achieve learning outcomes as successfully 
as students of traditional design-with-make activities? 

• What are student and teacher attitudes towards 
design-without-make activities within technology 
education? 

Research Hypotheses 

The following research hypotheses identify the study testing 
instruments and subsequent data analysis: 

HO,: There will be no significant difference in knowledge 
gain between the traditional design-with-make (control) 
and design-without-make (treatment) groups. 

H0 2 : There will be no significant difference in attitudes 
between the traditional design-with-make (control) and 
design-without-make (treatment) groups. 

H0 3 : There will be no significant difference in the 
effectiveness of the traditional design-with-make (control) 
and design-without-make (treatment) groups. 

Population Sample 

Due to the non-random sampling present in the 
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convenience sample taken, accurate generalization 
across the population was not possible, The sample 
chosen for this study consisted of two high school 
'Fundamentals of Technology 1 courses being taught by the 
same instructor, The classes consisted of a total of 27 
students, which is enough for statistical analysis to take 
place (Agresti & Finlay, 1997). 

Control Group 

For this study, after the pre-test, the control group received a 
standard course of instruction in design principles and 
elements, consisting of a design-with-make reinforcement 
activity. Instruction consisted of a PowerPoint lecture on the 
principles and elements of design, while the design-with- 
make activity was reserved for students to design and 
construct a 2D or 3D advertisement for the technology of 
their choice. The lesson plans and PowerPoint, as well as 
activity guidelines, can be found in the Appendix of 
Thompson, (2009), After completion of this course of 
instruction, students received the post-test and 
performance was evaluated. 

Experimental Group 

The experimental group also started with the pre-test, which 
was identical to the pre-test taken by the control group. 
Students then received the treatment, which consists of a 
PowerPoint lecture on the principles and elements of 
design which was geared towards the completion of a 
design-without-make activity. For the design-without-make 
experiment activity students were asked to brainstorm and 
research an emerging technology. They were then asked 
to design a new product for presentation to the class, in 
which the new technology they had researched could be 
used to solve a real-world problem they have identified. 
Lesson plans, PowerPoint, and activity guidelines, can be 
found in the Appendix of Thompson, (2009) an unpublished 
Masters Thesis, After students have completed their 
presentations to the class, they took the post-test and their 
performance was evaluated. Again, the post-test for the 
experimental group is identical to the post-test of the 
control group. 

Procedure 

Two high school Fundamentals of Technology classes were 
randomly chosen and parental permission obtained for 


subject participation. This study was planned around a set 
of strict guidelines and took place during the week of 
March 16, 2009. One week prior, the researcher and the 
cooperating teacher met for a 2 hour teacher training 
session on the treatment pedagogy. The researcher 
outlined the entire study, the new instructional method, and 
provided the teacher with a set of materials relating to the 
instructional unit and activities. The researcher also 
introduced the subjects to the study, 

Upon the start of this study, both groups took a pre-test. Next 
they were taught virtually identical lessons on the unit 
material, with the only difference being slides related to the 
activities students would be doing in conjunction with the 
lessons. After completing the lectures, students were given 
handouts corresponding to the activities designed for their 
particular group within the study. Subjects then used the 
remainder of the week of instruction to complete the 
assigned projects and present them to their peer group. 
Once the unit requirements and study time had been 
fulfilled, students took the post-test. Throughout the study 
duration, the researcher gathered observational data 
relating to student involvement and problems 
encountered during implementation of the treatment 
lesson and activity. 

At the end of this study, semi-structured interviews were 
conducted to gain an understanding of teacher and 
student attitudes towards design-without-make activities 
within the classroom. One teacher and two students 
involved in the design-without-make activity were 
interviewed. Specific questions asked of the teacher were: 

• What are your views on the quality of ideas produced? 

• Would you recommend design-without-make to other 
[technology] teachers? 

• Were the ideas produced creative? 

• Did pupils come to value the ideas as a product in 
themselves? 

• Did this strategy alienate pupils from the curriculum? 
(Barlex&Trebell, 2008, p. 124) 

The two student interviewees were asked: 

• Their thoughts of design-without-make. 

• What they actually designed? 
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• What they thought of their design? 

• Would they have had to design something simpler if 
that had to make it? 

• Would they recommend design-without-make as a 
means of enhancing design skills? 

• Did the unit lead to the production of creative ideas? 
(Barlex&Trebell, 2008, p, 124) 

Upon completion of the interviews, transcriptions were 
made and the data was collected into tables showing the 
question asked as well as the response given, The data was 
then analyzed and conclusions drawn as to whether a 
positive or negative attitude was shown towards design- 
without-make overall. 

Testing Instrumentation 

The pre-test and post-test were both derived from the non- 
secure test item bank that all North Carolina Fundamentals 
of Technology teachers are given with their curriculum 
guide for the course, Upon the completion of the item bank 
questions, the test items were evaluated by a panel of 
professors, teachers, industry professionals, and 
Department of Public Instruction officials to determine 
validity of each item (Shown, 2008). Furthermore, reliability 
tests were performed on the questions as they were pilot 
tested during development (Shown, 2008), However, to 
establish reliability of the assessments used within this study, 
a split-halves correlation was performed comparing 
individual student performance on similar questions 
between the pre- and post-tests. 

In developing these instruments from the overall item bank, 
the researcher isolated questions relating to the specific 
objectives involved in the research study unit. Then 
duplicate items were eliminated, Next, every other item 
was chosen for the pre-test and the remaining items were 
used in the post-test. Both tests were checked to 
approximate the consistency of items related to topics 
within the lesson and then one question was deleted from 
the pretest to make both tests of equal length for ease of 
comparison. 

Data Analysis 

After students took the pre-test and post-test, the 
assessments were scored. And an analysis of covariance 


was performed to help account for extraneous variables 
within the study. Upon completion of the covariance, an 
ANOVA statistical analysis was run to compare 
effectiveness of objective achievement for each group. 
These statistics were compared to determine whether a 
significant difference exists between groups and to see 
which hypotheses, if any, was correct. An ANOVA was 
chosen based upon the assumptions that the post-test 
score is normally distributed for each population as defined 
by the pre-test, the variances are the same for all 
populations, and that the sample is randomly chosen from 
the population (Green & Salkind, 2003, p. 161), 

The qualitative data was analyzed using keyword coding, 
which seeks to identify keywords indicating attitudes within 
the interview transcriptions. The keywords were also 
assessed based on context to determine whether they 
exude a generally positive or negative attitude towards the 
design-without-make process. Additionally, a frequency 
analysis was run for each question to determine the 
number of positive and negative comments given by the 
interviewees. These results were then grouped into charts 
and conclusions drawn as to student and teacher attitudes 
towards the design-without-make activity over all. 

Analysis 

Quantitative Analysis 

This analysis section is in two parts: i) Quantitative analysis 
using descriptive statistics of the gathered data, and ii) 
Qualitative analysis via brief interviews to teachers and 
students. 

A one-way analysis of variance was conducted to evaluate 
the relationship between instruction and the change in pre- 
and post-test scores between groups. The pre-test scores 
were set as between-subjects factors, because of the 
comparison of scores between subjects, with two levels (i.e. 
control and treatment). The post-test score was set as the 
dependent variable, because of that score's dependence 
upon the initial pre-test score and instructional methods 
used. Using SPSS statistical analysis software, the data was 
analyzed using a univariate linear model and then again 
using a comparison of means. Both tests yielded identical 
results. 

The study consisted of 27 participants, with the control 
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group having 10 and the treatment group having 1 7 (Table 
3). The mean score from both groups came to 14.37 
(SD=5.43), and a standard error of 1.05. As seen in Table 4, 
post-test scores ranged from 4 to 22 out of a possible score 


Variables 


Definition 

Groups 



Control 

Treatment 

Independent Variable 


Pretest Score 

Dependent Variable 


Posttest Score 

Means 



Pretest Posttest Difference 

Treatment 



10.88 14.71 3.82 

Control 



10.7 13.8 3.1 

Descriptive Statistics-95% Confidence Intervals of Pairwise Differences in 

Pretest and Posttest Scores between tests. 


Group 


M 

SD N 

Treatment 

Pretest 

10.88 

3.24 17 


Posttest 

14.71 

5.02 17 

Control 

Pretest 

10.70 

3.92 10 


Posttest 

13.80 

6.30 10 

Table 3. Descriptive Statistics 

Control Group 




ID# 

Pre 

Post 

Pre-Post Differnece 

1 

13 

14 

1 

2 

6 

16 

10 

3 

7 

4 

-3 

4 

8 

10 

2 

5 

11 

17 

6 

6 

9 

4 

-5 

7 

11 

12 

1 

8 

20 

22 

2 

9 

11 

19 

8 

10 

11 

20 

9 



Mean Difference : 3.1 

Treatment Group 




ID# 

Pre 

Post 

Pre-Post Differnece 

1 

6 

13 

7 

2 

12 

18 

6 

3 

14 

22 

8 

4 

9 

20 

11 

5 

6 

8 

2 

6 

14 

21 

7 

7 

13 

17 

4 

8 

5 

16 

11 

9 

17 

13 

-4 

10 

13 

8 

-55 

11 

8 

5 

-3 

12 

12 

14 

2 

13 

12 

13 

1 

14 

12 

22 

10 

15 

10 

13 

3 

16 

10 

11 

1 

17 

12 

16 

4 



Mean Difference : 3.823529 


Table 4. Raw Test Scores 


of 22. With F=2,04, p=.05, there were no significant 
differences between the control and treatment post-test 
scores. 

Qualitative Analysis 

At the conclusion of the study, brief interviews were 
conducted involving the teacher and two students, one 
male and one female, After the interviews, transcriptions 
were made and the data was analyzed using thematic 
coding based on the identification of keywords and 
phrases. The questions asked of the teacher, as well as the 
teacher's responses can be seen in Thompson (2009), with 
the student questions and answers being found in the 
Appendix, (Thompson, 2009), 

From this information, keywords/phrases were identified 
and coded as having a generally positive theme towards 
design-without-make, or a generally negative theme 
towards design-without-make. Some of the positive 
identifiers from the teacher interview were: much higher, 
not limited, choices, variety, use it all the time, advantage, 
like, conceptual, far greater, far better, input, and enjoyed. 
Some of the negative identifiers were: especially the one 
that were with the make, don't like, not as much, why didn't 
we get to build something?, disadvantage, non-made, 
actually fabricate, skills, like making, and enjoy making. 
Some of the positive identifiers from the student interviews 
were: interesting, I liked it, awesome, fun, worked out, 
different, good, nice, it could be useful, yes, and improve. 
Some of the negative identifiers were: bad group, 
difference, and didn't workout. 

After coding all of the data, frequencies were tabulated 
and attitudes were inferred from the frequency of positive 
to negative identifiers. It can be inferred from the data 
(Table 5) that the teacher had a generally positive view 
towards design-without-make, with a 2 positive comment 
to 1 negative comment ratio. It can also be inferred that 
the teacher can see both advantages and disadvantages 
to this methodology, but overall, they had a positive outlook 
for design-without-make. It can be inferred from the data 
(Table 6) that the students combined had a generally 
positive view towards design-without-make, with over 3 
positive comments to every 1 negative comment. It can 
also be inferred that Student 1 had a much more positive 
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Q1 

Q2 Q3 Q4 

Q5 

Q6 

Q7 

Q8 

Totals 

Positive 4 

2 2 3 

3 

3 

0 

3 

20 

Negative 0 

0 1 1 

3 

0 

3 

2 

10 

Table 5.Overall Responses: Teacher 

Overall Responses: Students (combined) 

Q1 Q2 Q3 Q4 

Q5 

Q6 

Q7 

Totals 

Positive 9 

0 3 

1 

2 

3 

0 

18 

Negative 3 

0 1 

1 

0 

0 

0 

5 

Overall Responses: Student 1 

Q1 Q2 Q3 

Q4 

Q5 

Q6 

Q7 

Totals 

Positive 4 

0 1 

1 

1 

1 

0 

8 

Negative 0 

0 0 

0 

0 

0 

0 

0 

Overall Responses: Student 2 

Q1 Q2 Q3 

Q4 

Q5 

Q6 

Q7 

Totals 

Positive 5 

0 2 

0 

1 

2 

0 

10 

Negative 3 

0 1 

1 

0 

0 

0 

5 


Table 6. Student Responses 


attitude towards design-without-make than did Student 2, 
although Student 2 still had an overall positive response with 
a positive to negative ratio of 2:1. 

Implications and Importance of this study 

Design-without-make has many advantages to the 
modern technology education teacher. It promotes 
creativity, builds teamwork skills, helps students become 
more innovative designers, and requires far fewer 
resources than the traditional design-with-make activities. 
According to the data collected in this study, the design- 
without-make pedagogy is just as, if not more, effective 
than traditional design-with-make methodologies, which 
makes this pedagogy a new tool for the technology 
teacher. While design-without-make should not completely 
replace design-with-make projects in technology 
education, as technology education moves towards more 
integration with other subject areas design-without-make 
may become an even more powerful resource to many, 

Recommendations for future research 

This study can be used as a precursor to a more in depth 
design-with-make project or another design-without-make 
activity. Either may help reiterate and solidify the 
importance of the design process within students' minds. 
Future research in this area might focus around: 

• Building stronger data sets using larger samples to test 
pedagogy effectiveness. 

• Multiple units of instruction to determine which topics 


are best taught using design-without-make 
methodologies. 

• Determining which type of teachers this pedagogy is 
best suited for. 

Conclusion 

From this research (Thompson, 2009), the authors 
described how many students in North Carolina are 
motivated to take technology education because of its 
hands-on, project-based approach, but they also point out 
how, these physical artifacts often lack any creativity or 
innovation on the part of the student, due to teacher 
designed plans. While these projects are easy to 
implement and fun for students to complete, often they are 
evaluated based on the completion of the product and an 
accompanying portfolio activity. By incorporating design- 
without-make projects in the place of traditional projects, 
many of these negatives can be avoided. However, if 
design-without-make activities are not helping students 
reach their standardized test score goals, then it is of no use 
to the modern technology education teacher who is 
revered or condemned based on those scores. 

From this study, it appears that design-without-make is as 
effective a tool for teaching design fundamentals to 
students as traditional design-with make activities. 
According to the data in this study (Thompson, 2009), there 
was no significant difference in student performance 
between the groups when run at a 95% confidence 
interval. Part of this can be explained by Badran (2007), 
who outlines that co-curricular activities, team work, 
diversified activities, and strong ties with industry are also 
important factors for developing creativity in the 
classroom. Creativity, which helps build intrinsic motivation 
within students (Spendlove, 2007), is becoming 
increasingly more important to the future because of the 
"unlimited horizons" it may open up, providing for ever- 
broadening, multidisciplinary creativity and innovation 
(Badran, 2007). Creativity is directly integrated into the 
Standards for Technological Literacy (STL), developed by 
the International Technology Education Association (ITEA), 
and is closely tied to the topics addressed in any 
technology education classroom. Standards and testing 
for standards help teachers ensure that students are 
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gaining the intelligence and knowledge base that 
Sternberg, Reznitskaya, and Jarvin (2007) talk about and, 
based upon the results of this study, design-without-make is 
an acceptable approach to teaching technology 
standards. 

References 

[1] , Agresti, A., & Finlay, B. (1997). Statistical methods tor 
the Social Sciences. Upper Saddle River, New Jersey: 
Prentice Hall. 

[2] , Albanese, M. A., & Mitchell, S. (1993). Problem-based 
learning: A review of literature on its outcomes and 
implementation issues. Academic Medicine, 68(1), 52-81. 

[3] , Atkinson, S. (2000). Does the Need for High Levels of 
Performance Curtail the Development of Creativity in 
Design and Technology Project Work? International Journal 
of Technology and Design Education, 10, 255-281. 

[4] , Badran, I. (2007). Enhancing creativity and innovation 
in engineering education. European Journal of 
Engineering Education, 32(5), 573-585. 

[5] , Bandura, A. (1997). Social Learning Theory. 
Englewood Cliffs, NJ: Prentice Hall. 

[6] , Banks, F. & Jackson, G. O. (2007). The role of making in 
design & technology. In Barlex, D. (Ed.), Design & 
technology for the next generation (pp. 186-197). 
Shropshire, England: Cliffe & Company Ltd. 

[7] , Barlex, D. (2007). Creativity in school design & 
technology in England: a discussion of influences. 
International Journal of Technology and Design Education, 
17,149-162. 

[8] , Barlex, D. M. &Trebell, D. (2008). Design-without-make: 
Challenging the conventional approach to teaching and 
learning in a design and technology classroom. The 
International Journal of Technology and Design Education, 
18,119-138. 

[9] , Berg, R., Pezalla-Granlund, M., Resnick, M., 8t Rusk, N. 
(2008). New pathways into robotics: Strategies for 
broadening participation. Journal of Science Education 
and Technology, 17,59-69. 

[10] , Brusic, S.A., Fales, J.F., 8r Kuetemeyer, V.F. (1999). 

Technology: Today and tomorrow. New York, NY: 
Glencoe/McGraw-Hill. 


[11] , Davydov, V. V. & Kerr, S. T. (1995). The influence of L. S. 
Vygotsky on education theory, research, and practice. 
Educational Researcher, 24(3), 12-21. 

[12] , Gredler, M.E. (2005). Learning and instruction: Theory 
into practice. Upper Saddle River, NJ: Pearson, Merrill, 
Prentice Hall. 

[13] , International Technology Education Association 
(ITEA). (2000). Standards for technological literacy. Reston, 
VA: ITEA. 

[14] , John-Steiner, V. (2000) Creative Collaboration. 
Oxford, Oxford University Press 

[15] , Kipperman, D. &. Sanders, M. (2007). Mind (not) the 
gap.. .Take a risk. In Barlex, D. (Ed.), Design & technology for 
the next generation (pp. 186-197). Shropshire, England: 
Cliffe &. Company Ltd. 

[16] , Lambros, A. (2002). Problem-based learning in K8 
classrooms: A teacher's guide to implementation. 
Thousand Oaks, CA. Corwin Press. 

[17] , Marton, F. &Saljo, R. (1976). On qualitative differences 
in learning - I: Outcome and process. British Journal of 
Educational Psychology 46, 4-11. 

[18] . Matthews, B, (2004). The Effects of Direct and 
Problem-based Learning Instruction in an Undergraduate 
Introductory Engineering Graphics Course. Unpublished 
Doctoral dissertation. NC State University. 

[19] , Mills, J.E. & Treagust, D.F. (2003). Engineering 
education-ls problem-based or project-based learning the 
answer? Australian Journal of Engineering Education. 

[20] , Murphy, R and Hennessy, S. (2001). Realizing the 
potential -and lost opportunities - for peer collaboration in 
a D and T setting. The International Journal of Technology 
and Design Education, 11, 203-237 

[21] , North Carolina Department of Public Instruction 
(NCDPI). (2006). Technology Education. Retrieved Oct. 31, 
2008, from http://www.dpi.state.nc.us/cte/technology/ 
index.html 

[22] , Peterson, R. E. (2001). Establishing the creative 
environment in Technology Education. The Technology 
Teacher, 61, 7-9. 

[23] , Schwartz, J. (2007, September 30). Re-engineering 


28 


i-manager’s Journal of Educational Technology, Vol. 7 • No. 1 • April - June 2010 






RESEARCH PAPERS 


engineering. TheNewYorkTimes, pp.6,94. 

[24] . Shown, T. (personal communication, November 7, 
2008). North Carolina Department of Public Instruction 
(NCDPI). Technology Education. Retrieved Oct. 31, 2008, 
from http://www.dpi.state.nc.us/cte/technology/index.html 

[25] , Spendlove, D. (2007). The locating of emotion within a 
creative, learning and product oriented design and 
technology experience: Person, process, product. The 
International Journal of Technology and Design Education, 
18, 45-57. 

[26] . Sternberg, R. J., Reznitskaya, A., & Jarvin, L. (2007). 

Teaching for wisdom: What matters is not just what students 
know, but how they use it. London Review of Education, 
5(2), 143-158, 

[27] , Thompson, T.B. (2009). Design without make: A 


feasible direction for American Technology Education. 
Unpublished Master's Thesis. NC State University. 

[28] , Thompson, Varnado, & Matthews. Design without 
make A new Design pedagogg for STEM Education. /- 
manager's Journal of Educational Technology, Vol. 6 Issue 
No. 1 April-June, 2009.pp. 20-27. 

[29] , Trebell, D. (2007, July). A literature review in search of 
an appropriate theoretical perspective to frame a study of 
designerly activity in secondary design and technology. 
Paper presented at the meeting of The Design and 
Technology Association Education & International 
Research Conference, Telford, England. 

[30] , Vygotsky, L.S. (1978). Mind in Society. Cambridge, 
MA: Harvard University Press. Downloaded from URL 
http://tip.psychology.org/vygotsky.html (March 1,2009) 


ABOUT THE AUTHOR 


Dr. Brian Matthews coordinated and developed the scientific visualization curriculum for the North Carolina State Department of 
public instruction (NC-SDPI], He is currently creating a new GC496 civil engineering graphics course for the civil engineering 
school-to start pilot courses. 



i-manager’s Journal of Educational Technology, Vol. 7 • No. 1 • April - June 2010 


29 







